"A fundamental problem in artificial intelligence is that nobody really knows what intelligence is." That's the opening sentence of "Universal Intelligence: A Definition of Machine Intelligence" (link) authored by Shane Legg and Marcus Hutter. If those names sound familiar, they are. Legg is a co-founder of DeepMind and Hutter is a senior scientist at DeepMind - two very well accomplished individuals with a long track record researching artificial general intelligence. While both have done exemplary work in this field, this paper, in my opinion, is poor.

When we talk about artificial intelligence (AI) – which we have done lot recently, including my outline on The Conversation of liability and regulation issues – what do we actually mean? AI experts and philosophers are beavering away on the issue. But having a usable definition of AI – and soon – is vital for regulation and governance because laws and policies simply will not operate without one. This definition problem crops up in all regulatory contexts, from ensuring truthful use of the term "AI" in product advertising right through to establishing how next-generation automated weapons systems (AWSs) are treated under the laws of war. True, we may eventually need more than one definition (just as "goodwill" means different things in different contexts). But we have to start somewhere so, in the absence of a regulatory definition at the moment, let's get the ball rolling.

A fundamental problem in artificial intelligence is that nobody really knows what intelligence is. The problem is especially acute when we need to consider artificial systems which are significantly different to humans. In this paper we approach this problem in the following way: We take a number of well known informal definitions of human intelligence that have been given by experts, and extract their essential features. These are then mathematically formalised to produce a general measure of intelligence for arbitrary machines. We believe that this measure formally captures the concept of machine intelligence in the broadest reasonable sense.

A fundamental problem in artificial intelligence is that nobody really knows what intelligence is. The problem is especially acute when we need to consider artificial systems which are significantly different to humans. In this paper we approach this problem in the following way: We take a number of well known informal definitions of human intelligence that have been given by experts, and extract their essential features. These are then mathematically formalised to produce a general measure of intelligence for arbitrary machines. We believe that this equation formally captures the concept of machine intelligence in the broadest reasonable sense. We then show how this formal definition is related to the theory of universal optimal learning agents. Finally, we survey the many other tests and definitions of intelligence that have been proposed for machines.

This paper is a survey of a large number of informal definitions of ``intelligence'' that the authors have collected over the years. Naturally, compiling a complete list would be impossible as many definitions of intelligence are buried deep inside articles and books. Nevertheless, the 70-odd definitions presented here are, to the authors' knowledge, the largest and most well referenced collection there is.